Multiuser Co-Inference With Batch Processing Capable Edge Server

Graphics processing units (GPUs) can improve deep neural network inference throughput via batch processing, where multiple tasks are concurrently processed. We focus on novel scenarios that the energy-constrained mobile devices offload to an edge server with GPU. The task is partitioned into sub-tasks for a finer granularity of offloading and scheduling, user energy consumption minimization problem under latency constraints investigated. To deal coupled scheduling introduced by concurrent we first consider offline constant same constraint. It proven optimizing policy each independently aggregating all in one optimal, thus independent partitioning sub-task (IP-SSA) algorithm inspired. Further, optimal grouping (OG) proposed optimally group when different. Finally, future arrivals cannot be precisely predicted, deterministic gradient (DDPG) agent trained call OG. Experiments show IP-SSA reduces up 94.9% setting, while DDPG-OG outperforms DDPG-IP-SSA 8.92% online setting.